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Four  dimensions^had  previously  been  extracted  ih  a  multidimensional 
scaling  analysis  of  the  Naval  aviation  electronics  technician's  job  at  the  entry 

level.  Wt  F sent  unidimensional' Scales  < 

on  each  orthese  dimmieionst  utilising  mettTOtTS  developed 

For  each  dimension,  a  set  of  tasks  was  found  which 


formed  a  scale  as  defined  by  both  the  Thurstone  method  of  equal  appearing  in 
tervals  and  the  Guttman  technique.  (  J 
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CHAPTER  1 


INTRODUCTION 

The  feasibility  of  applying  multidimensional  scaling  techniques  for  job 
analysis  and  job  descriptive  purposes  has  been  examined  in  three  previous 
Applied  Psychological  Services'  studies  (Schultz  and  Siegel,  1962;  Siegel  and 
Schultz,  1963;  Schultz  and  Siegel,  1963b).  These  were  the  most  recent  efforts 
in  a  progressive,  systematic  attack  on  problems  in  the  development  of  job  pro¬ 
ficiency  criteria  for  graduates  of  Naval  training  programs  in  several  technical 
specialties.  Earlier  studies  (Schultz  and  Siegel,  1961;  Siegel  and  Schultz, 

1962)  had  produced  methods  for  constructing  job  task  performance  criterion 
instruments  which  meet  the  Thurstone  and/or  Guttman  scalability  requirements  . 
The  research  described  in  this  report  related  these  two  areas  of  development 
in  order  to  obtain  a  new  type  of  criterion  measurement  instrument  for  use  in 
the  Navy. 

Multidimensional  scaling  analysis  has  as  its  principal  purpose  the 
determination  of  the  number  of  basic  dimensions  (factors)  underlying  a  set  of 
perceptions.  The  characteristics  of  and  names  for  the  extracted  dimensions 
should  be  apparent  from  inspection  of  the  scale  values  of  the  stimuli  on  each 
axis.  In  the  first  application  of  multidimensional  scaling  to  job  tasks,  Schultz 
and  Siegel  (1962)  studied  the  job  of  the  striker  and  petty  officer,  third  class, 
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Naval  aviation  electronics  technician  (AT).  Four  underlying  factors  were 
found  to  account  for  this  job  as  seen  by  immediate  supervisors  in  the  rating. 

The  dimensions  were  called  "electro-comprehension,  "  "equipment  operation 
and  inspection  (routine),  "  "electro-repair  (simple),  "  and  "electro-safety.  " 

The  report  described  the  dimensions  as  "reasonable  and  meaningful"  and  went 
on  to  point  out  that: 

Furthermore,  these  four  dimensions  appear 
to  be  amenable  to  unidimensional  scaling,  i.  e. ,  they 
possess  characteristics  which  would  seem  to  make  it 
possible  to  develop  unidimensional  scales  to  measure 
each  of  them.  Since  the  dimensions  extracted  in  the 
present  study  represent  the  underlying  structure  of 
the  job  performed  by  striker  and  third  class  AT's,  as 
perceived  by  their  supervisors,  the  next  logical  step 
in  measuring  that  job  performance  would  appear  to  be 
the  construction  of  unidimensional  scales  on  each  of 
the  dimensions . If  unidimensional,  scaled  instru¬ 

ments  are  constructed,  it  will  then  be  possible  to 
evaluate  the  job  performance  of  individuals  on  each  of 
the  orthogonal  dimensions  seen  by  AT  supervisors  as 
composing  the  job.  (Schultz  and  Siegel,  1962,  p.  36). 

As  mentioned  above,  methods  for  measuring  job  performance  on  a 
Thurstone  or  a  Guttman  type  of  scale  resulted  from  previous  work  carried  out 
at  Applied  Psychological  Services.  Those  studies  (Schultz  and  SiegrT,  1961; 
Siegel  and  Schultz,  1962)  involved  the  twin  steps  of:  (1)  demonstrating  that 
job  task  skills  are  scalable  according  to  the  definitions  of  scalability  of  each 
of  the  two  approaches,  and  (2)  developing  check  lists  of  job  tasks  in  a  form  that 
could  be  applied  easily  and  economically  in  the  evaluation  of  the  job  performance 
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of  individuals.  Use  of  the  check  lists,  which  met  the  Thurstcne  and  the  Guttman 
scalability  requirements,  called  for  simple,  t  .vo-category  responses  to  clearly 
and  directly  stated  questions. 

Because  of  the  manner  in  which  the  directions  and  questions  were  formu¬ 
lated  in  those  first  studies  of  Thurstone  and  Guttman  scales  of  job  performance, 
scores  on  the  scaled  check  lists  reflected  the  general  developmental  stage  or 
level  of  the  Naval  technician  being  evaluated.  There  appeared  to  be  no  reason, 
however,  why  the  same  methods,  slightly  adapted,  could  not  be  applied  to  each 
one  of  the  dimensions  extracted  in  a  multidimensional  scaling  analysis  of  a  job. 


Purposes  of  the  Present  Study 

The  purposes  of  the  present  study  were  to:  (1)  investigate  the  feasibility 
of  developing  unidimensional  scales  along  each  of  the  dimensions  extracted  in  a 
multidimensional  scaling  analysis  of  job  performance,  and  (2)  produce  practical 
scales,  meeting  the  Thurstone  and  Guttman  scalability  requirements,  which 
could  be  used  for  evaluating  individual  technicians  along  the  four  dimensions 


previously  found  to  underlie  the  job  of  the  striker  and  the  petty  officer,  third 


class.  Naval  aviation  electronics  technician. 


CHAPTER  II 


DEVELOPMENT  OF  THURSTONE  EQUAL  APPEARING 
INTERVAL  SCALES  FOR  JOB  PERFORMANCE  DIMENSIONS 

The  principal  output  of  a  multidimensional  scaling  analysis  is  a  matrix 
of  the  scale  values  (projections)  of  each  stimulus  on  each  dimension  of  a  set  of 
stimuli.  The  present  study  was  to  build  upon  the  results  of  the  analysis  by 
Schultz  and  Siegel  (1962)  of  the  jobs  of  the  striker  and  the  petty  officer,  third 
class,  Naval  aviation  electronics  technician.  Therefore,  the  starting  point  was 
a  matrix  of  the  loadings  of  18  tasks  on  each  of  four  dimensions,  (The  matrix 
can  be  found  in  Table  8,  page  2  1,  of  the  Schultz  and  Siegel  report,  ) 

Several  different  approaches  were  possible  to  the  construction  of  uni- 
dimensional  scales  along  the  four  dimensions.  One,  analogous  to  the  develop¬ 
ment  of  "pure  factor"  tests,  would  have  called  for  writing  four  new  series  of  job 
tasks,  each  series  to  include  only  tasks  which  uere  thought  to  represent  varying 
amounts  of  one  dimensional  variable  but  not  to  involve  the  other  dimensions.  This 
approach,  although  possible,  appeared  to  represent  a  more  formidable  under¬ 
taking  than  the  one  adopted.  As  an  alternative  approach,  the  original  tasks 
could  be  used,  even  though  they  all  tended  to  be  loaded  on  more  than  one  dimen¬ 
sion.  In  this  approach,  the  subjects  of  the  study  who  would  be  making  scale 
judgments  would  be  asked  to  view  a  complex  stimulus,  i.  e.  ,  a  job  task,  from  the 
standpoint  of  its  relationship  to  each  dimension  separately.  If  this  method  proved 
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to  be  feasible,  it  would  have  the  further  advantage  of  indicating  whether  the 
stimulus  scale  positions  on  each  dimension  as  determined  by  the  direct  percep¬ 
tions  of  judges  are  closely  related  to  the  scale  values,  i.  e.  ,  the  projections,  of 
the  stimuli  on  that  dimension,  as  it  emerged  from  the  multidimensional  scaling 
analysis.  It  was  decided  to  take  the  latter  approach  because:  (1)  it  did  not  in¬ 
volve  the  burden  of  writing  new  task  descriptions  in  "pure"  dimensional  form, 
and  (2)  it  seemed  more  likely  to  produce  a  fruitful  result. 

The  Technical  Task  Evaluation  List 

Review  of  the  18  job  tasks  found  to  constitute  the  job  of  the  striker  and 
petty  officer,  third  class,  aviation  electronics  technician  suggested  that  the  re¬ 
lationship  of  some  of  the  tasks  to  certain  of  the  dimensions  would  be  so  abstruse 
or  30  very  obvious  (e.  g.  ,  "using  proper  safety  precautions  for  self"  in  relation 
to  "electro-safety")  as  simply  to  confuse  the  subjects  and  serve  only  to  interfere 
with  the  proper  placement  of  the  remaining  tasks  along  the  particular  continuum. 
Therefore,  for  three  of  the  dimensions,  several  tasks  were  eliminated  from 
further  consideration.  The  tasks  included  for  analysis  on  each  dimension  are 
indicated  by  X’s  in  the  appropriate  columns  of  Table  1 . 

The  Technical  Task  Evaluation  List  was  then  constructed  for  the  purpose 
of  obtaining  the  judgments  required  in  developing  Thurstone  equal- appearing  in¬ 
tervals  scales.  Each  of  the  sections  pertained  to  one  of  the  four  dimensions 
being  treated.  The  over-all  directions,  printed  on  the  cover  page,  were: 
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There  are  four  sections  in  this  form,  each 
containing  a  number  of  tasks  which  are  customarily 
done  by  AT  strikers.  You  will  be  asked  to  evaluate 
the  tasks  listed  in  a  section  from  a  certain  point  of 
view,  but  this  point  of  view  will  differ  from  one 
section  to  another.  Therefore,  be  sure  you  under¬ 
stand  how  you  are  to  evaluate  the  tasks  each  time. 

Read  the  directions  for  each  section  very  carefully, 
even  though  there  is  some  repetition  in  them. 

In  each  section  the  respondent  was  given  a  continuum  with  eleven  equally 
spaced  points  indicated  (except  for  the  third  section  which  involved  only  nine 
points).  The  dimension  under  consideration  was  briefly  defined  and  the  re¬ 
spondent  was  asked  to  judge  each  of  the  listed  tasks  only  from  the  standpoint  of 
that  dimension.  The  low,  middle,  and  high  areas  on  the  continuum  were  describ¬ 
ed.  The  judge  was  asked  to  indicate  the  placement  of  each  task  on  the  continuum 
by  means  of  gummed,  prenumbered  stickers,  which  were  distributed  symmetri¬ 
cally  with  generally  larger  frequencies  in  the  middle  categories.  The  stickers 
were  used  in  order  to  force  the  judges  to  distribute  their  responses  widely  and 
more  or  less  normally.  In  each  section  the  tasks  were  listed  in  an  order  deter¬ 
mined  from  random  numbers  tables;  furthermore,  the  order  was  varied  from 
one  section  to  another. 

The  "directions  page"  for  the  first  section,  pertaining  to  electro- 
comprchension,  is  presented  in  Table  2. 
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Performing  variety  oi  "housekeeping"  duties  such  as 


Operating  standard  test  equipment  for  determining  mal¬ 
functions  in  avionic  equipments 


TABLE  2 


Directions  Page  for  Section  1  of  Technical  Task  Evaluation  List 


and  undpr,SteCH0n  1  18  tasks  which  Cal1  for  various  amounts  of  knowledge 

^r:  d;r  »Lr,h ciples  of  ,a:ioni°  "rr"i,rv  «««■ 

may  wr„e  your  responses  in  with  a  pencil  and  then  go  back  and  attach  the  stickers. 


ACQUIRES  LITTLE  KNMA.COQC 
AND  UNDERSTAND INQ  OF  PRIN- 

cin.cs  or  avionic  circuitry 


REQUIRE*  MODERATE  KNOWLEDGE 
AND  UNDERSTAND INQ  OF  PR|N- 

cmE*  or  avionic  circuitry 


REQUIRES  GREAT  KNOWLEDGE 
AND  UNDERSTANDING  or 
PR  INC I RLE*  Or  AVIONIC 
CIRCUITRY 


1.  First  place  the  sticker  with  the  U_  on  it  next  to  the  task  that 
requires  the  greatest  knowledge  and  understanding  of  the 
principles  of  avionic  circuitry. 

2.  Then  pick  the  task  that  requires  the  next  greatest  knowledge 
and  understanding  of  avionic  circuitry  and  put  the  sticker  with 
the  on  it  next  to  that  task. 

3.  Now  find  the  task  that  falls  in  category  "nine"  and  place  the 
sticker  with  th"  £  on  it  next  to  that  task. 

4.  Next  find  the  task  that  requires  the  least  knowledge  and  under- 
standing  of  the  principles  of  avionic  circuitry.  Put  the  stic  ker 
with  the _1_  on  it  next  to  this  task. 

5.  Which  task  requires  the  next  smallest  knowledge  and  understand¬ 

ing  of  avionic  circuitry?  Put  the  sticker  with  the  2  on  it  next  to 
this  task.  — 

6.  Put  the  sticker  with  the  3  on  it  next  to  the  task  that  falls  in  cate¬ 
gory  three. 

7.  Now  place  the  remaining  tasks  in  their  proper  categories  by  ap- 
olymg  the  remaining  stickers  <4,  5,  6,  7,  8). 


11 


8 


In  these  directions  the  respondent  was  asked  to  judge  each  of  the  tasks 
listed  from  the  standpoint  of  the  extent  to  which  it  requires  knowledge  and  under¬ 
standing  of  the  principles  of  avionic  circuitry.  The  directions  for  the  tasks  re¬ 
lating  to  the  second  and  third  extracted  dimensions  (operation  and  inspection, 
and  electro-repair)  preceded  the  second  and  third  sections  of  the  form.  These 
directions  asked  the  respondent  for  estimates  of  the  level  of  ability  required  for 
each  of  the  listed  tasks  in  terms  of  level  of  ability  in  equipment  turn-on,  warm¬ 
up,  run,  and  shut-down  procedures  and  in  terms  of  level  of  ability  in  the  removal 
of,  making  required  corrections  in,  and  the  replacement  of  avionic  equipment. 
The  fourth  and  last  sections  of  the  form,  relative  to  electro- safety,  asked  for  an 
estimate  of  the  extent  to  which  safety  precautions  are  employed  in  performing 
each  of  the  listed  tasks. 

The  complete  Technical  Task  Evaluation  List  is  shown  in  Appendix  A  to 
this  report. 

Sample 

The  Technical  Task  Evaluation  List  was  administered  to  40  chief  petty 
officers  and  petty  officers,  first  class,  in  the  Naval  aviation  electronics  techni¬ 
cian  (AT)  rating.  *  The  squadrons  to  which  these  judges  were  assigned  and  their 

At  the  time  of  responding,  two  of  the  subjects  were  in  the  anti¬ 
submarine  warfare  technician  (AX)  rating,  a  new  rating  for  electronics  tech¬ 
nicians  specializing  in  that  area  of  work.  Each  of  these  men  had  previously 
been  an  AT  for  5  years. 
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locations  are  presented  in  Table  3.  Their  average  age  was  31.  3  years  and 

they  had  12.  4  years  of  military  experience,  on  the  average.  For  approximately 

eight  years,  they  had  been  AT's  and  had  been  assigned  during  this  time  to  an 
average  of  3.  2  squadrons. 

Administration 

Two  groups,  one  at  each  of  the  two  Naval  Air  Stations  involved,  were 
assembled  for  administration  of  the  Technical  Task  Evaluation  List.  In  order 
to  make  certain  thai  the  men  read  the  directions  to  each  section  and  that  they 
understood  what  they  were  to  do,  the  administrator  kept  the  group  together,  i.  e. , 
he  read  the  directions  for  a  section  to  the  group,  asked  for  questions,  and  then 
had  everyone  wait  until  all  the  men  were  finished  with  that  section  before  pro¬ 
ceeding  to  the  next  section.  Completion  of  the  form  consumed  about  one  hour. 
Responding  to  the  first  section,  containing  the  most  tasks,  seemed  to  take  quite 
a  while,  but  the  later  sections  went  more  quickly. 

From  the  purely  mechanical  point  of  view,  the  administration  of  the 
Technical  Task  Evaluation  List  seemed  to  proceed  well.  Only  one  person  be¬ 
came  confused  writing  in  the  numbers  before  affixing  the  numbered  stickers,  a 
procedure  suggested  in  the  directions.  The  few  niioctmno  tU  _ _  i 

- ~ 77  i**v«**«Mu  me  rcaponaenis 

asked  involved  the  meaning  of  the  task  descriptions.  The  group  gave  the 
appearance  of  understanding  and  being  able  to  accomplish  its  assignments;  in¬ 
formal  conversations  with  some  of  the  subjects  after  the  sessions  supported 
this  view. 
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Table  3 


Number  of  Supervisors,  by  Location  and 
Squadron,  Completing  Technical  Task  Evaluation  List 


Location 


Squadron 


Number 


Norfolk 


Oceana 


FAETULANT 
HS  3 
HS  7 
HU  2 
VAW  12 
VP  56 
VRC  40 
VKF  31 
VS  24 
VS  26 
VS  27 
VS  36 
VU  6 


2 

2 

2 

1 

2 

2 

2 

2 

1 

2 

2 

2 

2 


VA  42 
VA  81 
VA  83 
VA  85 
VF  74 
VF  103 
VU  2 
VU  4 


2 

2 

3 

1 

3 

1 

2 

2 


40 
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Results 


In  order  for  a  scale  to  achieve  the  requirements  of  a  Thurstone  scale, 
equal- appearing  between-scale  value  distances  are  sought.  To  establish  a 
scale  by  the  Thurstone  method  of  equal-appearing  intervals,  the  median  and 
semi -interquartile  range  of  the  judges'  decisions  are  taken  as  the  scale  and 
deviation  (Q)  values  of  the  stimuli.  Therefore,  for  each  task  in  each  section  of 
the  Technical  Task  Evaluation  List,  these  two  statistics  were  computed.  Plots 

of  the  resulting  data  for  the  four  sections  are  presented  in  Figures  1  2,  3,  and 

4. 

Examination  of  the  four  plots  reveals,  first  of  all,  that  in  each  case  the 
full  range  of  the  scale  was  utilized.  The  scale  values  are  distributed  well  over 
the  scales  and  the  highest  and  lowest  values  generally  come  closer  to  the  ex¬ 
tremes  of  the  scales  than  had  been  true  in  earlier  work  on  scaling  tasks  (Schultz 
and  Siegel,  1961;  Siegel  and  Schultz,  1962).  The  Q  values  are,  for  the  most 
part,  fairly  small,  indicating  relative  agreement  among  the  judges  as  to  the  task 
placement  on  the  scales  and  suggesting  the  probability  of  good  discrimination 
among  the  tasks.  There  is  some  slight  tendency  for  the  Q  values  to  be  higher 
for  the  tasks  with  higher  S  values  in  Sections  2  a-d  4  of  .he  Technical  Task 
Evaluation  List. 
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CURE  2.  SCALE  VALUE  (S)  AND  SEMI  INTERQUARTILE  RANGE  (Q)  FOR  EACH  TASK 
IN  SECTION  2  OF  TECHNICAL  TASK  EVALUATION  I  1ST 


o 
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FIGURE  4  SCALE  VALUE  (S)  AND  SEMI  INTtRQUARTlLE  RANGE  (Q)  FOR  EACH  TASK 
IN  SECTION  4  OF  TECHNICAL  TASK  EVALUATION  LIST 


Correlation  of  Obtained  Scale  Values 
with  Task  Loadings  on  Each  Dimension 

For  each  task  included  in  a  section  of  the  Technical  Task  Evaluation  List 
two  numbers  representing  the  position  of  that  task  on  the  dimension  involved  had 
been  obtained.  On  the  one  hand,  the  loading  derived  from  the  multidimensional 
scaling  analysis  (Schultz  and  Siegel,  1962,  p.  28)  was  the  projection  of  the  task 
on  the  dimension.  On  the  other  hand,  the  Thurstone  scale  value  produced  in  this 
study  grew  out  of  an  appraisal  of  the  task  as  it  related  to  the  defined  dimension. 
The  former  constituted  the  results  of  an  indirect  approach,  i.  e. ,  an  analysis  of 
over-all  judgments  of  the  similarity  among  all  the  original  tasks.  The  latter 
was  a  direct  placement  of  each  task  along  a  defined  scale.  The  two  resultant 
numbers  should  be  highly  correlated  provided:  (1)  the  dimensions  produced  in 
the  multidimensional  scaling  analysis  are  meaningful  and  interpretable,  (2)  the 
descriptions  of  the  dimensions  in  the  Technical  Task  Evaluation  List  are  accu¬ 
rate  and  complete  reflections  of  the  dimensional  characteristics, and  (3)  each  of 
the  approaches  gives  stable  indices  of  the  task  scale  placements. 

For  each  section  of  the  Technical  Task  Evaluation  List,  i.  e. ,  for  each 
dimension,  the  Pearson  product-moment  correlation  coefficient  was  computed 

between  the  two  sets  of  scale  values  described  above.  The  results  were  as 
follows: 
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Dimension 


Electro-comprehension  (18  tasks) 
Operation  and  inspection  (12  tasks) 
Electro-repair  (9  tasks) 
Electro-safety  (13  tasks) 


In  each  case,  the  rank  of  at  least  one  or  two  tasks  was  noticeably  different  in 
the  two  sets  of  data  being  correlated.  In  the  "electro-repair"  section,  the  task 
of  "operating  standard  test  equipment  for  determining  malfunctions  in  avionic 
equipments"  was  given  the  highest  Thur stone  scale  value  but  was  seventh  in  the 
group  of  nine  tasks  from  the  standpoint  of  dimensional  loadings.  In  the  "electro- 
safety"  section,  there  were  three  instances  of  large  variations:  "standing-watch" 
was  first  in  dimensional  loading  and  eleventh  in  Thurstone  scale  value;  "per¬ 
forming  preventive  maintenance  on  avionic  equipments"  was  tenth  in  dimensional 
loading  and  second  in  Thurstone  scale  value;  and  "performing  intermediate  in¬ 
spections  of  avionic  equipments"  was  eleventh  in  dimensional  loading  and  fifth  in 
Thurstone  scale  value.  These  relatively  large  differences  undoubtedly  played  a 
major  role  in  reducing  the  correlation  coefficients  for  the  sections  involved  as 
compared  w’lth  the  values  which  might  have  been  other  wise  obtained,  since  the 
N’s  on  which  the  computations  were  based  were  not  large. 

Over-all,  the  correlation  coefficients  given  above  lend  considerable  sup¬ 
port  to  a  contention  supporting  the  validity  of  multidimensional  scaling  analytic 
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techniques  when  they  are  applied  to  job  task  data.  i.  e. .  the  scaling  methods 
produce  meaningful  results  which  can  be  verified  from  other  points  of  view. 
Since  similar  scale  values  were  obtained  for  most  of  the  tasks  studied  in  the 
two  approaches,  the  exceptions  are  probably  most  satisfactorily  explained  in 
terms  of  the  construction  of  the  Technical  Task  Evaluation  List,  rather  than 
in  terms  of  the  inadequacies  of  multidimensional  s.  sling  methodology  or  the 
instability  of  the  results.  A  plausible  explanation  of  the  discrepancies,  for 
example,  is  that  all  of  the  details  of  the  underlying  dimensions  extracted  in 
the  multidimensional  scaling  analysis  were  not  clearly  defined  and/or  some  of 
the  more  refined  aspects  of  the  dimensions  were  no,  fully  communicated  to  the 
men  who  made  the  Thurstone  scale  judgments.  The  high  level  of  the  correlation 
coefficients,  however,  indicates  that,  for  the  most  part:  (1)  the  dimensions  are 
meaningful.  (2)  the  nature  of  the  dimensions  was  accurately  represented  in  the 
Technical  Task  Evaluation  List,  and  (3)  the  scale  values  produced  by  both 
analytical  methods  are  reliable. 

Item  Selection  for  Thurstone  Type  Sc  alas 

The  construction  of  a  Thurstone  type  of  scale  requires  a  series  of  items 
which,  on  the  one  hand,  have  a  wide  scattering  of  S  values  ,ha.  ,kc  c_.,r_ 

continuum  is  represented  and,  on  the  other  hand,  have  small  Q  values  so  that 
they  overlap  minimally  in  perceived  scale  placement.  Within  each  section  of 
the  Technical  Task  Evaluation  List,  a  selection  of  items  was  made  to  meet  these 
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two  requirements  as  closely  as  possible.  The  items  constituting  the  Thurstone 
scale  for  each  dimension,  i.  e. ,  each  Evaluation  List  section,  are  presented  in 
Tables  4-7,  along  with  their  S  and  Q  values.  It  is  thus  demonstrated  that  it  is 
possible  to  builo  unidimensional  Thurstone  scales  along  each  of  the  dimensions 

extracted  in  a  multidimensional  scaling  analysis  when  the  stimuli  consist  of  job 
tasks. 
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Table  4 


S  (Scale)  and  Q  (Semi-Interquartile  Range)  Values  of  Items 
Selected  for  Thurstone  Scale  on  Dimension  1,  Electro-comprehension 

Task 

_S 

Q 

Standing  watch 

1.  11 

.  31 

Performing  variety  of  "housekeeping" 
duties  such  as  cleaning  shop,  repairing 
tools,  etc. 

2.  04 

.  39 

Using  proper  safety  precautions  for  self 

3.  62 

1. 30 

Replacing  repaired  avionic  parts/equip¬ 
ment  in  planes 

4.  88 

.  81 

Performing  minor  inspections  of  avionic 
equipments 

5.  81 

.86 

Using  inspection  and  operation  manuals 

6.  88 

1 .  13 

Performing  intermediate  inspections  of 
avionic  equipments 

7.  68 

.  82 

Performing  preventive  maintenance  on 
avionic  equipments 

8.  03 

.  88 

Operating  standard  test  equipment  for 
determining  malfunctions  in  avionic 
equipments 

9.  81 

.  72 

Using  schematics  for  standard  circuits 
in  avionic  equipments 

10.  67 

.  54 
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Table  5 


S  (Scale)  and  Q  (Semi-Interquartile  Range)  Values  of  Items  Selected 
forTliurstone  Scale  on  Dimension  2,  Equipment  Operation  and  Inspection 


Task 

_S_ 

Q_ 

Standing  watch 

1. 00 

.  25 

Making  out  reports  (failure,  etc.  ) 

2.  33 

1. 01 

Removing  malfunctioning  avionic  parts/ 
equipment  from  planes 

3.  50 

.  81 

Replacing  repaired  avionic  parts/equipment 
in  planes 

3.  96 

.  90 

Performing  postflight  inspections  of  avionic 
equipments 

6.  18 

.  92 

Performing  preflight  inspections  of  avionic 
equipments 

7.  20 

1.  12 

Performing  preventive  maintenance  on 
avionic  equipments 

8.  62 

1.45 

Operating  avionic  equipments 

10.  06 

1.00 

Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 

10.  33 

1.49 

Table  6 


S  (Scale)  and  Q  (Semi-Interquartile  Range)  Values  of  Items 
Selected  for  Thurstone  Scale  on  Dimension  3,  Electro- repair 


Task 

_S_ 

R. 

Standing  watch 

1. 00 

.  25 

Performing  routine  line  operations 

2.  83 

1. 85 

Performing  postflight  inspections  of 
avionic  equipments 

4.  03 

.  76 

Performing  preflight  inspections  of 
avionic  equipments 

4.  32 

1. 00 

Performing  minor  inspections  of  avionic 
equipments 

5.  21 

1.  31 

Performing  intermediate  inspections  of 
avionic  equipments 

7.  50 

1. 00 

Performing  preventive  maintenance  on 
avionic  equipments 

7.  73 

.  94 

Operating  standard  test  equipment  for 
determining  malfunctions  in  avionic 
equipments 

8.  21 

1. 08 
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Table  7 


S  (Scale)  and  Q  (Semi-Interquartile  Range)  Values  of  Items 
Selected  for  Thurstone  Scale  on  Dimension  4,  Electro-safety 


Task 

_S_ 

Q 

Making  out  reports  (failure,  etc.  ) 

1.  33 

.  51 

Following  block  diagrams  for  avionic 
equipments 

2.  17 

.  56 

Standing  watch 

2.  83 

1 . 00 

Performing  minor  inspections  of  avionic 
equipments 

5.  68 

.  88 

Replacing  repaired  avionic  parts/equipment 
in  planes 

5.  79 

.  88 

Performing  preflight  inspections  of  avionic 
equipments 

6.  39 

1.25 

Performing  intermediate  inspections  of 
avionic  equipments 

7.  50 

1.54 

Performing  routine  line  operations 

8.  00 

2.30 

Performing  preventive  maintenance  on 
avionic  equipments 

8.  77 

I  .  37 

Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 

9.  93 

2.  29 
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CHAPTER  III 


DEVELOPMENT  OF  GUTTMAN 
SCALES  FOR  JOB  PERFORMANCE  DIMENSIONS 

The  application  to  the  job  task  performance  area  of  the  Guttman  method 
for  scaling  items  has  been  investigated  and  demonstrated  in  Applied  Psycho¬ 
logical  Services'  studies  previously  mentioned  (Schultz  and  Siegel,  1961;  Siegel 
and  Schultz,  1962).  The  technique  involves  testing  the  item  response  data  to 
determine  whether  a  hierarchy  exists  among  the  respondents  such  that  each 
person's  over-all  position  can  be  inferred  from  the  highest  item  in  the  set  to 
which  he  answers  affirmatively.  If  such  an  order  can  be  established,  the  items 
are  said  to  scale  and  it  can  be  assumed  that  an  individual  giving  an  affirmative 
response  to  one  item  will  have  a  high  probability  of  giving  an  affirmative  re¬ 
sponse  to  all  the  items  ranking  "lower"  in  the  set.  Since  the  earlier  studies 
concluded  that  it  is  possible  to  scale  job  tasks  by  the  Guttman  method,  the  ques¬ 
tion  under  investigation  in  the  current  study  was  whether  Guttman  scales  could 
be  constructed  along  the  dimensions  which  grew  out  of  a  multidimensional  scal¬ 
ing  analysis. 

The  Proficiency  Check  List 

I.  >  ■  - 

A  Guttman  scale  analysis  requires  data  in  terms  of  the  responses  of 
individuals  to  the  scale  items.  In  the  present  case,  this  meant  that  a  form  was 
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needed  which  would  permit  the  evaluation  of  individuals  rather  than  of  tasks. 

The  Proficiency  Check  List  was  designed  to  accomplish  this  purpose. 

In  certain  respects,  the  Proficiency  Check  List  was  similar  to  the 
Technical  Task  Evaluation  List  described  in  the  previous  Chapter.  Each  of 
the  four  sections  was  related  to  one  of  the  dimensions  produced  in  the  multi¬ 
dimensional  scaling  analysis  of  the  job  of  the  striker  and  petty  officer,  third 
class,  Naval  aviation  electronics  technician  (Schultz  and  Siegel,  1962).  Also, 
only  those  items  indicated  in  Table  1  of  this  report  were  included,  as  in  the 

i 

Technical  Task  Evaluation  List,  to  avoid  unnecessary  confusion.  In  a  more 
general  sense,  the  Proficiency  Check  List  took  the  same  approach  as  the 
Technical  Task  Evaluation  List  in  that  task  descriptions  as  they  appeared  in 
the  multidimensional  scaling  analysis  work  were  used,  rather  than  "pure  di- 

j 

mensional"  task  descriptions.  In  this  approach,  the  man  being  evaluated  was 

| 

to  be  judged  on  his  competency  on  only  those  aspects  of  a  task  that  were  re¬ 
lated  to  a  single  dimension  at  a  time. 

The  directions  on  the  cover  page  of  the  Proficiency  Check  List  asked 

S 

the  man  completing  the  form  to: 

Think  of  an  AT  striker  whom  you  have  super¬ 
vised  and  whose  capabilities  you  know  well.  You  are 
to  evaluate  him  in  four  different  ways.  There  are 
four  sections  in  this  form,  each  containing  a  number 
of  tasks  which  are  customarily  done  by  AT  strikers. 

You  will  be  asked  to  evaluate  the  man  you  are  con¬ 
sidering  with  respect  to  the  various  tasks  from  a  cer¬ 
tain  point  of  view,  but  this  point  of  view  will  differ 
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from  one  section  to  another.  Therefore,  be  sure 
you  understand  how  you  are  to  do  the  evaluations 
each  time.  Read  the  directions  for  each  section 
very  carefully,  even  though  there  is  some  repe¬ 
tition  in  them. 

Write  down  a  response  for  every  task.  If 
the  man  you  are  evaluating  has  not  done  a  task  or 
if  you  have  not  seen  him  do  it,  try  to  make  your 
best  estimate  of  the  evaluation. 

The  instructions  for  each  section  first  gave  a  brief  explanation  of  the 
dimension  under  consideration  (without  naming  it)  and  pointed  out  that  the  tasks 
included  in  the  list  for  that  section  involved  the  dimensional  characteristics  to 
a  varying  degree.  In  evaluating  the  man,  the  respondent  was  to  view  each  task 
only  from  the  standpoint  of  that  dimension.  Specifically,  the  directions  for 
Section  1  were: 


In  Section  1  there  are  18  tasks  which  call  for 
various  amounts  of  knowledge  and  understanding  of 
the  principles  of  avionic  circuitry.  You  should  think 
of  the  man  you  are  evaluating.  Does  he  have  the  know¬ 
ledge  and  understanding  of  the  principles  of  avionic 
circuitry  required  by  the  first  task?  If  he  does,  make 
a  check  in  the  box  under  "Yesk';  if  he  does  not,  make  a 
check  in  the  box  under  "No.  "  Then  look  at  the  second 
task  and  indicate  your  evaluation  of  the  man  by  a  check 
mark  in  the  appropriate  box.  Continue  on  down  the  page 
making  a  check  mark  in  one  box  next  to  each  task.  Re¬ 
member,  you  are  to  evaluate  the  man  ONLY  with  respect 
to  whether  or  not  he  has  the  knowledge  and  understanding 
of  the  principles  of  avionic  circuitry  required  by  each 
task.  Even  if  he  has  not  done  the  task  or  if  you  have  not 
seen  him  do  it  ,  make  your  best  estimate  of  his  capability 
with  regard  to  it.  Be  sure  to  make  a  check  mark  next  to 
each  task  in  the  list. 


Similarly,  Sections  2,  3,  and  4  respectively  asked  the  respondent  to 
judge  whether  the  man  he  evaluated  is: 

a.  capable  of  performing  proficiently  the  equip¬ 
ment  turn-on,  warm-up,  run,  and  shut-down 
procedures  involved  in  each  task 

i 

b.  capable  of  performing  proficiently  the  removal 
of,  making  required  corrections  in,  and  the  re¬ 
placement  of  avionic  equipment  involved  in  each 
task 

c. ,  capable  of  performing  the  safety  precautions  in¬ 

volved  in  each  task  listed. 

Within  the  sections,  the  tasks  were  presented  in  a  random  order  which 
was  different  for  each  section  and  all  of  which  differed  from  the  orders  used 
in  the  Technical  Task  Evaluation  List.  Thus,  the  respondent  was  not  expected 
to  "read  into"  the  list  any  underlying  scale  as  a  result  of  a  systematic  order  of 
presentation. 

The  complete  Proficiency  Check  List  is  given  in  Appendix  B  of  this  re¬ 


port. 


Sample 

A  group  of  28  strikers  and  petty  officers,  third  class,  in  the  Naval 
aviation  electronics  technician  rating  were  evaluated  on  the  Proficiency  Check 
List.  The  squadrons  to  which  these  men  were  assigned  and  their  locations  are 
presented  in  Table  8. 
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TABLE  8 


Number  of  Technicians,  by  Location  and 
Squadron,  Evaluated  on  Proficiency  Check  List 


Location 

Squadron 

Number 

Patuxent  River 

AEWTULANT 

8 

VP  8 

5 

VR  1 

3 

Argentia  (Newfoundland) 

VW  11 

l 

VW  13 

A 

1 

Bermuda 

VP  49 

4 

Brunswick 

VP  7 

1 

Moffett  Field 

VR  8 

1 

Naples  (Italy) 

VR  24 

A 

28 
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Administration 


The  subjects  of  this  experiment  were  evaluated  by  14  chief  petty  officers 
and  petty  officers,  first  class,  in  the  AT  rating  who  were  located  at  Patuxent 
River  Naval  Air  Station  and  who  were  assigned  to  one  of  the  following  squadrons: 
AEWTULANT,  VP  8,  VP  49,  and  VR  1.  The  Proficiency  Check  List  was  ad¬ 
ministered  to  the  entire  group  at  one  time.  None  of  the  other  forms  described 
in  this  report  were  completed  by  these  men. 

The  raters  were  asked  to  select  a  man  who  was  one  of  the  poorer  workers 
they  had  supervised.  They  reported  having  known  the  ratees  an  average  of  13.  8 
months,  with  a  range  of  from  3  to  36  months. 

The  average  age  of  the  raters  was  30.  8  years  and  they  had  an  average  of 
13.  2  years  of  military  service.  In  general,  they  had  been  AT's  for  10.  7  years 
and  been  assigned  to  4.  4  squadrons  as  AT's. 

No  problems  were  encountered  in  the  administration  of  the  Proficiency 
Check  List. 

Analytic  Method 

Following  the  procedure  of  the  earlier  scaling  studies,  the  method  of 
Guttman  analysis  described  by  Green  (1956)  was  employed.  In  this  method  an 
index  of  consistency,  I,  is  used  to  replace  the  several  requirements  for  seal- 
ability  proposed  by  Guttman.  I  relates  the  obtained  reproducibility  to  that 


expected  by  chance.  Green  suggests  that  I  should  be  .  50  or  higher  in  order  to 
consider  the  set  of  items  a  scale  in  the  Guttman  sense,  although  he  points  out 
there  is  no  rationale  for  setting  any  rigid  cutting  point. 

As  in  the  previous  work,  the  sets  of  tasks  to  be  tested  for  Guttman  seal- 
ability  consisted  of  the  tasks  selected  for  the  Thurstone  scales.  That  is,  only 
the  tasks  selected  for  inclusion  in  the  Thurstone  scale  for  each  dimension  (as 
described  in  Chapter  II  of  this  report)  were  used  in  the  analysis  of  that  dimen¬ 
sion.  As  a  result,  the  numbers  of  tasks  in  Sections  1-4  of  the  Proficiency  Check 
List  for  which  respondent  data  were  involved  in  the  present  analysis  were  10,  9, 

8.  and  10,  respectively.  In  effect,  the  Thurstone  analysis  functioned  as  a  first 
step  in  the  Guttman  analysis. 

Results 

The  results  of  the  Guttman  scalability  analysis  for  the  four  dimensions 
are  presented  in  Table  9.  All  four  I  values  are  well  above  .  50.  This  suggests 

that  scales  which  meet  Green's  criterion  of  scalability  in  the  Guttman  sense 
have  been  achieved. 
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TABLE  9 


Results  of  Guttman  Scalability  Analysis  for 
the  Four  Sections  of  the  Proficiency  Check  List 


Section  Number  and  Dimension  Name 

1.  Electro-comprehension  {Knowledge 
and  understanding  of  the  principles 
of  avionic  circuitry) 

2.  Equipment  operation  and  inspection 
(Equipment  turn-on,  warm-up,  run, 
and  shut-down  procedures) 

3.  Electro-repair  (Removal  of,  making 
the  required  corrections  in,  and  re¬ 
placement  of  avionic  equipment) 

4.  Electro-safety  (Safety  precautions) 


Rep*  Repj** 


.957  .887 

.968  .901 

.  969  .  880 

.  975  . 890 


*  Reproducibility 

**  Reproducibility  exacted  by  chance 
***  Index  of  consistency 


I*** 


.  619 

.  677 

.  742 
.  773 
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CHAPTER  IV 


DISCUSSION 


The  results  presented  in  Chapters  II  and  III  tend  to  indicate  that  it  is 
possible  for  judges  to  view  job  tasks  and  technicians  from  several  different  view¬ 
points  and  to  make  meaningful  ratings  of  the  tasks  and  people  on  the  various  di¬ 
mensions  involved.  Apparently  judges  are  able  to  change  their  set  in  response  to 
rather  simple  and  straightforward  instructions.  Scales  were  produced  which  call 
for  merely  a  "yes"  or  a  "no"  response  to  a  small  number  of  items,  so  that  the 
scales  should  be  convenient  and  practical  to  use  in  the  Fleet. 

It  is  quite  possible,  of  course,  that  unidimensional  scales  could  be  de¬ 
veloped  by  some  other  approach  to  the  construction  of  the  scale  items.  It  might 
be  profitable,  for  example,  to  attempt  to  construct  scales  of  a  "pure  factor"  type 
and  to  compare  the  utility  of  scales  produced  through  that  approach  with  the  scales 
produced  in  the  current  study. 

Now  that  Thurstone  and  Guttman  type  scales  are  available  for  each  of  the 
orthogonal  dimensions  extracted  in  a  multidimensional  scaling  analysis,  the  ques¬ 
tion  may  be  raised  as  to  how  the  scales  should  be  weighted  if  they  are  to  be  com¬ 
bined  in  an  over-all  evaluation  of  a  man.  Solution  to  this  problem  is  not  merely 
a  matter  of  putting  back  together  what  has  been  taken  apart  by  multidimensional 
scaling,  since  there  are  many  bases  for  deriving  weights  The  matter  of  weight¬ 
ing  job  components  has  been  mentioned  as  a  major  problem  area  in  a  recern  survey 


-  33  - 


of  problems  and  progress  in  job  performance  measurement  (Schultz  and  Siegel, 
1963a):  In  the  present  case,  it  would  seem  that  some  kind  of  importance  values 
for  each  of  the  dimensions  might  provt  useful. 

The  present  scale  developmental  study  was  based  on  the  results  of  a 
previous  Applied  Psychological  Services'  study  (Siegel  and  Schultz,  1962)  in 
which  it  was  found  that,  at  least  for  the  job  performance  domain,  the  establish¬ 
ment  of  a  Thurstone  scale  represents  a  reasonable  and  efficient  first  step  in  the 
development  of  a  Guttman  scale.  The  results  of  the  present  study  lend  further 
support  to  this  previous  finding. 

Other  open  questions  involve  how  and  whether  the  scales  might  be  em¬ 
ployed  for  providing  information  on  the  technical  proficiency  of  the  fleet.  Un¬ 
answered  also  is  the  question  of  the  correlation  of  proficiency  ratings  obtained 
by  the  current  method  with  ratings  obtained  through  other  evaluative  techniques, 
such  as  work  sample  performance  tests,  which  possess  higher  face  validity. 


CHAPTER  V 


t== 


SUMMARY  AND  CONCLUSIONS 

Summary 

Applied  Psychological  Services  has  been  making  a  progressive,  system¬ 
atic  attack  on  a  series  of  related  problems  in  the  development  of  job  proficiency 
criteria  for  graduates  of  Naval  training  programs  in  several  technical  special¬ 
ties.  Earlier  studies  (Siegel  and  Schultz,  1962)  investigated  the  hypothesis  that 
job  performance  skills  are  scalable  in  the  same  manner  as  attitudes  and  sensory 
phenomena.  The  methods  developed  in  those  studies  for  producing  job  perform¬ 
ance  scales  which  meet  Thurstone  and/or  Guttman  requirements  were  tested  for 
applicability  across  several  related  job  specialties  (Schultz  and  Siegel,  1961). 
Attention  was  then  turned  to  the  analysis  of  job  performance  through  the  appli¬ 
cation  of  multidimensional  scaling  techniques  (Schultz  and  Siegel,  1962;  Siegel 
and  Schultz,  1963;  Schultz  and  Siegel,  1963b). 

The  present  study  attempted  to  bring  these  previous  developments  to¬ 
gether  in  order  to  provide  practical  personnel  evaluation  instruments  for  the 
independent  dimensions  underlying  a  technical  job.  Specifically,  the  study  was 
designed  to;  (1)  investigate  the  feasibility  of  developing  unidimensional  scales 
along  each  of  the  dimensions  extracted  in  a  multidimensional  scaling  analysis  of 
job  performance  and  (2)  produce  practical  scales,  meeting  the  Thurstone  and  the 
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Guttman  scalability  requirements,  which  could  be  used  for  evaluating  individual 
technicians  along  the  four  dimensions  previously  found  to  underlie  the  job  of  the 
striker  and  the  petty  officer,  third  class,  Naval  aviation  electronics  technician. 

Starting  with  18  tasks  found  to  constitute  the  job  of  the  striker  and  the 
petty  officer,  third  class,  Naval  aviation  electronics  technician  (AT)  (Schultz 
and  Siegel,  1962),  two  forms  were  developed  and  administered  to  groups  of  AT 
supervisors.  The  first  form  asked  the  supervisors  to  judge  each  task  from  the 
standpoint  of  its  relationship  to  each  of  the  four  underlying  job  dimensions  sep¬ 
arately.  The  second  form  requested  evaluations  of  individual  technicians  on  the 
tasks,  as  the  tasks  were  viewed  from  the  standpoint  of  each  dimension. 

For  each  job  dimension,  a  subset  of  tasks  was  found  which  scaled  accord¬ 
ing  to  the  Thurstone  equal  appearing  intervals  method.  These  groups  of  tasks 
also  met  the  requirements  of  Guttman  scales.  Support  for  the  validity  of  multi¬ 
dimensional  scaling  techniques,  as  applied  in  the  job  task  area,  was  indicated  by 
generally  high  correlation  coefficients  obtained  between  the  direct  task  ratings 
on  each  dimension  obtained  in  this  study  and  the  task  loadings  on  each  dimension 
obtained  in  a  multidimensional  scaling  analysis  of  the  job  involved. 

Conclusions 

It  seems  reasonable  to  arrive  at  the  following  conclusions  from  the  ma¬ 
terial  presented  in  this  report: 
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1.  It  is  possible  to  develop  unidimensional  scales 
along  each  of  the  job  dimensions  extracted  in  a 
multidimensional  scaling  analysis. 


2.  Scales  meeting  the  Thurstone  and  Guttman  seal- 
ability  requirements,  which  were  developed  in 
this  study,  can  be  used  to  evaluate  individual 
technicians  along  the  four  dimensions  underlying 
the  job  of  striker  and  the  petty  officer,  third  class. 
Naval  aviation  electronics  technician. 

3.  The  application  of  multidimensional  scaling  methods 
to  a  job  task  constellation  produces  meaningful,  re¬ 
liable  results  which  can  be  verified  by  other  experi¬ 
mental  procedures. 
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APPENDIX  A 

Appendix  A  presents  the  Technical  Task 
Evaluation  List  employed  in  the  study  here  reported 
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TECHNICAL  TASK  EVALUATION  LIST 


Name _  Today's  Date _ 

Squadron _  Naval  Air  Station  At _ _ 

Your  Rating _  Your  Age _  Years  in  Service _ 

What  is  the  total  number  of  years  you  have  been  an  AT? _ years 

How  many  different  squadrons  have  you  been  assigned  to  as  an  AT?  _ 


Directions 

There  are  four  sections  in  this  form,  each  containing  a  number  of  tasks 
which  are  customarily  done  by  AT  strikers.  You  will  be  asked  to  evaluate  the 
tasks  listed  in  a  section  from  a  certain  point  of  view,  but  this  point  of  view  will 
differ  from  one  section  to  another.  Therefore,  be  sure  you  understand  how  you 
are  to  evaluate  the  tasks  each  time.  Read  the  directions  for  each  section  very 
carefully,  even  though  there  is  some  repetition  in  them. 


PREPARED  BY 

APPLIED  PSYCHOLOGICAL  SERVICES 
WAYNE,  PENNSYLVANIA 

UNDER  CONTRACT  NONR  22T9(00) 

WITH  THE 

OFFICE  OF  NAVAL  RESEARCH 
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Directions  for  Section  1 


In  Section  1  there  are  18  tasks  which  call  for  various  amount  of  knowledge 
and  understanding  of  the  principles  of  avionic  circuitry.  First,  you  should  read 
over  the  tasks  to  see  what  they  are.  Then,  using  the  scale  below  as  a  guide,  you 
are  to  judge  each  task  ONLY  from  the  standpoint  of  the  knowledge  and  understanding 
o .if  the  principles  of  avionic  circuitry  that  it  requires.  Follow  the  procedure  de¬ 
scribed  in  the  steps  given  beloyr.  If  you  wish,  before  you  use  the  stickers  you  may 
write  your  responses  in  with  a  pencil  and  then  go  back  and  attach  the  stickers. 


1  2  3  4  5  6  7  8 


9  10  11 


REQUIRE*  LITTLE  KNOWLEDGE 
AND  UNDERSTAND! NO  OF 
PRINCIPLES  OP  AVIONIC 
CIRCUITRY 


REQUIRES  MODERATE  KNOWLEDGE 
AND  UNDERSTANDING  OP 
PRINCIPLES  OF  AVIONIC  CIR¬ 
CUITRY 


REQUIRES  GREAT  KNOWLEDGE 
AND  UNDERSTAND I NO  OF 
PRINCIPLES  OF  AVIONIC 
CIRCUITRY 


1.  First  place  the  sticker  with  the  11  on  it  next  to  the  task  that 
requires  the  greatest  knowledge  and  understanding  of  the 
principles  of  avionic  circuitry. 

2.  Then  pick  the  task  that  requires  the  next  greatest  knowledge 
and  understanding  of  avionic  circuitry  and  put  the  sticker 
with  the  1£  on  it  next  to  that  task. 

3.  Now  find  the  task  that  falls  in  category  "nine"  and  place  the 
sticker  with  the  j)  on  it  next  to  that  task. 

4.  Next  find  the  task  that  requires  the  least  knowledge  and  under¬ 
standing  of  the  principles  of  avionic  circuitry.  Put  the  sticker 
with  the  on  it  next  to  this  task. 

5.  Which  task  requires  the  next  smallest  knowledge  and  under¬ 
standing  of  avionic  circuitry?  Put  the  sticker  with  the  2  on 
it  next  to  this  task. 

6.  Put  the  sticker  with  the  3^  on  it  next  to  the  task  that  falls  in 
category  '  three.  " 

7.  Now  place  the  remaining  tasks  in  their  proper  categories  by 
applying  the  remaining  stickers  (4,  5,  6,  7,  8). 


pq 

10 

9 

8 

8 

7 

7 

6 

□ 

E 

6 

5 

5 

4 

4 

3 

2 

zj 
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SECTION  1 


To  what  extent  is  knowledge  and  understanding  of 
the  principles  of  avionic  circuitry  required  in 


Scale  Value 


1.  Replacing  repaired  avionic  parts /equipment  in  planes 

1 

2.  Following  block  diagrams  for  avionic  equipments 

3.  Using  safety  precautions  on  equipments 

4.  Performing  intermediate  inspections  of  avionic  equip¬ 
ments 

5.  Using  inspection  and  operation  manuals 

6.  Removing  malfunctioning  avionic  parts  /equipment  from 
planes 

7.  Operating  standard  test  equipment  for  determining  mal¬ 
functions  in  avionic  equipments 

8.  Performing  preflight  inspections  of  avionic  equipments 

9.  Performing  routine  line  operations 

10.  Performing  preventive  maintenance  on  avionic  equip¬ 
ments 

11.  Performing  postflight  inspections  of  avionic  equipments 

12.  Performing  variety  of  "housekeeping"  duties  such  as 
cleaning  shop,  repairing  tools,  etc. 

13.  Operating  avionic  equipments 

14.  Using  schematics  for  standard  circuits  in  avionic 
equipments 

15.  Making  out  reports  (failure,  etc.  ) 

16.  Performing  minor  inspections  of  avionic  equipments 

17.  Standing  watch 

18.  Using  proper  safety  precautions  for  self 
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Directions  for  Section  2 


In  Section  2  there  are  12  tasks  which  require  various  levels  of  ability  in 
equipment  turn-on,  warm-up,  run,  and  shut-down  procedures.  Although  the 
same  tasks  were  all  included  in  the  list  you  judged  in  Section  1,  this  time  you 
are  to  evaluate  them  from  the  standpoint  of  equipment  turn-on,  warm-up,  run, 
and  shut-down  procedures. 

First,  you  should  read  over  these  tasks  to  see  what  they  are.  Then, 
using  the  scale  shown  below  as  a  guide,  you  are  to  judge  each  of  these  tasks 
ONLY  from  the  standpoint  of  the  level  of  ability  in  equipment  turn-on,  Varm-up, 
run,  and  shut-down  procedures  which  it  requires.  Follow  the  procedure  de¬ 
scribed  in  the  steps  given  below.  Again,  if  you  find  it  helpful,  you  may  write  in 
your  responses  with  pencil  and  then  go  back  and  attach  the  stickers  over  your 
penciled  numbers. 


12  3  4 

REQUIRES  LOWEST  ABILITY 
LEVEL  IU  EQUIPMENT  TURN-ON, 
WARM-UP,  RUN,  AND  SHUT¬ 
DOWN  PROCEDURES 


5  6  7  8 

REQUIRES  MODERATE  ABILITY 
LEVEL  IN  EQUIPMENT  TURN¬ 
ON,  WARM-UP,  RUN  AND  SHUT¬ 
DOWN  PROCEDURES 


9  10  11 

REQUIRES  HIGHEST  ABILITY 
LEVEL  IN  EQUIPMENT  TURN¬ 
ON,  WARM-UP,  RUN,  AND 
SHUT-DOWN  PROCEDURES 


1.  First,  place  the  sticker  with  the  H.  on  it  next  to  the  task  that 
requires  the  highest  level  of  ability  in  equipment  turn-on, 
warm-up,  run,  and  shut-down  procedures. 

2.  Next,  find  the  task  that  involves  the  next  highest  level  of 
ability  in  equipment  turn-on,  warm-up,  run,  and  shut-down 
procedures  and  put  the  sticker  with  the  10_on  it  next  to  that 
task. 

3.  Now  find  the  task  that  falls  in  category  "nine"  and  place  the 
sticker  with  the_9  on  it  next  to  that  task. 

4.  Then  look  for  the  task  that  requires  the  lowest  level  of  ability 
in  equipment  turn-on,  warm-up,  run,  and  shut-down  pro¬ 
cedures.  Put  the  sticker  with  the  1  on  it  next  to  this  task. 

5.  Which  task  requires  the  next  lowest  level  of  ability  in  equip¬ 
ment  turn-on,  warm-up,  run,  and  shut-down  procedures? 

Put  the  sticker  with  the  2  on  it  next  to  this  task. 

6.  Put  the  sticker  with  the_3  on  it  next  to  the  task  that  falls  in 
category  "three". 

7.  Now  place  the  remaining  tasks  in  their  proper  categories  by 
applying  the  remaining  stickers  (4,  5,  6,  7,  8). 


SECTION  2 


What  level  of  ability  in  equipment  turn-on,  warm-up, 
run,  and  shut-down  procedures  is  requii  eel  in 


Scale  Value 


1.  Performing  minor  inspections  of  avionic  equipments 

2.  Operating  avionic  equipments 

3.  Performing  routine  line  operations 

4.  Standing  watch 

5.  Performing  preventive  maintenance  on  avionic  equip¬ 
ments 

6.  Performing  intermediate  inspections  of  avionic  equip¬ 
ments 

7.  Performing  preflight  inspections  of  avionic  equipments 

8.  Removing  malfunctioning  avionic  parts /equipment 
from  planes 

9.  Operating  standard  test  equipment  for  determining 
malfunctions  in  avionic  equipments 

10.  Performing  postflight  inspections  of  avionic  equipments 

11.  Making  out  reports  (failure,  etc. ) 

12.  Replacing  repaired  avionic  parts  /equipment  in  planes 
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Directions  for  Section  3 


This  time  you  are  to  evaluate  many  of  the  same  tasks  from  another  point 
of  view.  You  are  to  think  of  the  9  tasks  in  Section  3  in  terms  of  the  removal  of, 
making  required  corrections  in,  and  replacement  of  avionic  equipment  involved. 
The  question  you  should  ask  yourself  about  each  task  is,  "What  level  of  ability 
in  the  removal  of,  making  the  required  corrections  in,  and  the  replacement  of 
r  -ionic  equipment  is  required  in  this  task?" 

Again  you  should  first  read  over  these  tasks  to  see  what  they  are.  Then, 
using  the  scale  shown  below  as  a  guide,  you  are  to  judge  each  of  these  tasks 
ONLY  from  the  standpoint  of  the  removal  of,  making  required  corrections  in 
and  the  replacement  of  avionic  equipment. 

Follow  the  same  procedure  you  used  in  the  previous  sections.  That  is, 
first  select  the  three  tasks  which  require  the  highest  level  of  ability  in  the 
removal  of,  making  the  required  corrections  in,  and  the  replacement  of  avionic 
equipment  and  place  stickers_9,  8,  and  7  next  to  them.  Next,  find  the  three 
tasks  which  require  the  lowest  level  and  place  stickers  1,  2,  and  3  next  to  them. 
Then,  use  the  remaining  stickers  to  indicate  your  judgement  of  the  remaining 
tasks. 


12  3  4 


REQUIRES  LOWEST  LEVEL  OF 
ABILITY  IN  THE  REMOVAL  OF, 
MAKING  REQUIRED  CORRECTIONS 
IN,  AND  THE  REPLACEMENT  OF 
AVIONIC  EQUIPMENT 


5  6  7 


REQUIRES  MODERATE  LEVEL 
OF  ABILITY  IN  THE  REMOVAL 
OF,  MAKING  REQUIRED  COR¬ 
RECTIONS  IN,  AND  THE  RE¬ 
PLACEMENT  OF  AVIONIC 
EQUIPMENT 


89  10  11 

REQUIRES  HIGHEST  LEVEL  OF 
ABILITY  IN  THE  REMOVAL  OF, 
MAKING  REQUIRED  CORRECTIONS  IN, 
ANO  THE  REPLACEMENT  OF  AVIONIC 
EQUIPMENT 


EH 

m 

m 
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GH 

CD 

EH 

□ 

□ 
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SECTION  3 


What  level  of  ability  in  the  removal  of,  making  the  re¬ 
quired  corrections  in,  and  the  replacement  of  avionic 
equipment  is  required  in 


Scale  Value 


1.  Performing  postflight  inspections  of  avionic  equipments 

2.  Performing  preflight  inspections  of  avionic  equipments 

3.  Operating  avionic  equipments 

4.  Performing  preventive  maintenance  on  avionic  equip¬ 
ments 

5.  Operating  standard  test  equipment  for  determining  mal¬ 
functions  in  avionic  equipments 

6.  Performing  routine  line  operations 

7.  Performing  intermediate  inspections  of  avionic  equip¬ 
ments 

8.  Performing  minor  inspections  of  avionic  equipments 

9.  Standing  watch 
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Directions  for  Section  4 


The  Last  group  of  13  tasks,  shown  in  Section  4,  is  to  be  evaluated  from 
the  point  of  view  of  the  extent  to  which  the  employment  of  safety  precautions  is 
involved.  Read  over  the  sks  in  the  list  and  then,  using  the  scale  shown  below 
as  a  guide,  judge  each  one  ONLY  from  the  standpoint  of  the  extent  to  which  safety 
precautions  are  employed. 

Use  the  same  procedure  as  you  did  in  the  previous  sections.  Start  by 
finding  the  three  tasks  which  involve  the  greatest  employment  of  safety  pre¬ 
cations  and  place  stickers  11,  _1£,  and_9  next  to  them.  Next,  place  stickers  1, 

2,  and  3  next  to  the  three  tasks  which  involve  the  least  employment  of  safety 
precautions.  Last,  use  the  remaining  stickers  to  indicate  your  judgments  of  the 
remaining  tasks. 


12  3  4 

INVOLVES  LEAST  EMPLOYMENT 
OF  SAFETY  PRECATIONS 


5  6  7  T 

INVOLVES  MODERATE  EMPLOY¬ 
MENT  OF  8AFETY  PRECATIONS 


9  10  11 

INVOLVES  GREATEST  EMPLOY¬ 
MENT  OF  8AFETY  PRECAUTIONS 


SECTION  4 


To  what  extent  are  safety  precautions  employed  in 


Scale  Value 


1.  Performing  preflight  inspections  of  avionic  equip- 
ments 

2.  Making  out  reports  (failure,  etc. ) 

3.  Performing  preventive  maintenance  on  avionic 
equipments 

4.  Performing  postflight  inspections  of  avionic  equip¬ 
ments 

5.  Operating  avionic  equipments 

6.  Removing  malfunctioning  avionic  parts  /equipment 
from  planes 

7.  Performing  minor  inspections  of  avionic  equipments 

8.  Standing  watch 

9.  Performing  routine  line  operations 

10.  Performing  intermediate  inspections  of  avionic 
equipments 

11.  Operating  standard  test  equipment  for  determining 
malfunctions  in  avionic  equipments 

12.  Following  block  diagrams  for  avionic  equipments 

13.  Replacing  repaired  avionic  parts /equipment  in  planes 
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APPENDIX  B 


Appendix  B  presents  the  Proficiency  Check 
List  employed  in  the  study  here  reported 
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PROFICIENCY  CHECK  LIST 


Your  Name 


Today's  Date 


Directions 


Think  of  an  AT  striker  whom  you  have  supervised  and  whose  capabilities 
you  know  well.  You  are  to  evaluate  him  in  four  different  ways.  There  are  four 
sections  in  this  form,  each  containing  a  number  of  tasks  which  are  customarily 
done  by  AT  strikers.  You  will  be  asked  to  evaluate  the  man  you  are  considering 
with  respect  to  the  various  tasks  from  a  certain  point  of  view,  but  this  point  of 
view  will  differ  from  one  section  to  another.  Therefore,  be  sure  you  understand 
how  you  are  to  do  the  evaluations  each  time.  Read  the  directions  for  each  section 
very  carefully,  even  though  there  is  some  repetition  in  them. 

Write  down  a  response  for  every  task.  If  the  man  you  are  evaluating  has 
not  done  a  task  or  if  you  have  not  seen  him  do  it,  try  to  make  your  best  estimate 
of  the  evaluation. 


Name  of  Man  You  are  Rating _ 

His  Squadron _  Naval  Air  Station  At_ 

His  Rating _ 

How  long  have  you  known  him?  _ months 


PREPARE0  BY 

APPLICO  PSYCHOLOGICAL  SERVICE* 
WAYNE,  PENNSYLVANIA 

UNDER  CONTRACT  NONA  22?)(00  ) 

WITH  THE 

orricE  or  naval  research 
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Directions  for  Section  1 


i 
£ 

i 

j 

f 

In  Section  1  there  are  18  tasks  which  call  for  various  amounts  of  know¬ 
ledge  and  understanding  of  the  principles  of  avionic  circuitry.  You  should  think 
of  the  man  you  are  evaluating.  Does  he  have  the  knowledge  and  understanding  of 
the  principles  of  avionic  circuitry  required  by  the  first  task?  If  he  does,  make 
a  check  in  the  box  under  "Yes1';  if  he  does  not,  make  a  check  in  the  box  under 
"No.  "  Then  look  at  the  second  task  and  indicate  your  evaluation  of  the  man  by  a 
check  mark  in  the  appropriate  box.  Continue  on  down  the  page  making  a  check 
mark  in  one  box  next  to  each  task.  Remember,  you  are  to  evaluate  the  man 
ONLY  with  respect  to  whether  or  not  he  has  the  knowledge  and  understanding  of 
the  principles  of  avionic  circuitry  required  by  each  task.  Even  if  he  has  not 
done  the  task  or  if  you  have  not  seen  him  do  it,  make  your  best  estimate  of  his 
capability  with  regard  to  it.  Be  sure  to  make  a  check  mark  next  to  each  task  in 
the  list. 
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SECTION  1 


Does  he  have  the  knowledge  and  understanding  of 
the  principles  of  avionic  circuitry  required  by 


Yes  No 


1.  Performing  variety  of  "housekeeping"  duties 
such  as  cleaning  shop,  repairing  tools,  etc. 

2.  Following  block  diagrams  for*  avionic  equip¬ 
ments 

3.  Replacing  repaired  avionic  parts  /equipment 
in  planes 

4.  Making  out  reports  (failure,  etc. ) 

5.  Performing  preflight  inspections  of  avionic 
equipments 

6.  Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 

7.  Standing  watch 

8.  Using  proper  safety  precautions  for  self 

9.  Using  safety  precautions  on  equipments 

10.  Performing  intermediate  inspections  of 
avionic  equipments 

11.  Operating  avionic  equipments 

12.  Using  inspection  and  operation  manuals 

13.  Performing  routine  line  operations 

14.  Performing  minor  inspections  of  avionic 
equipments 

15.  Removing  malfunctioning  avionic  parts /equip¬ 
ment  from  planes 

16.  Using  schematics  for  standard  circuits  in 
avionic  equipments 

17.  Performing  postflight  inspections  of  avionic 
equipments 

18.  Performing  preventive  maintenance  on 
avionic  equipments 
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Directions  for  Section  2 


In  Section  2  there  are  12  tasks  which  involve  equipment  turn-on,  warm¬ 
up,  run,  and  shut-down  procedures  to  a  varying  degree.  Although  the  same 
tasks  were  all  included  in  the  list  in  Section  1,  this  time  you  are  to  view  them 
from  the  standpoint  of  equipment  turn-on,  warm-up,  run,  and  shut-down  pro 
cedures. 

Think  of  the  man  you  are  evaluating.  Is  he  capable  of  performing  pro¬ 
ficiently  the  equipment  turn-on,  warm-up,  run,  and  shut-down  procedures  in¬ 
volved  in  the  first  task  non  his  own'1  without  direct  supervision?  Ii  he  is,  make 
a  check  mark  in  the  column  under  ^Yes";  if  he  is  not,  make  a  check  mark  in  the 
column  under  "No.  "  Then,  look  at  the  second  task  and  indicate  your  evaluation 
nf  the  man  by  a  check  mark  in  the  appropriate  column.  Continue  on  down  the 
page.  For  each  task,  evaluate  the  man  ONLY  with  respect  to  whether  or  not  he 
is  capable  of  performing  proficiently  the  equipment  turn-on,  warm-up,  run,  and 
shut-down  procedures  involved  in  that  task  "on  his  own"  without  direct  super  - 
vision.  Make  a  check  mark  next  to  each  task,  estimating  his  capability  if  he  has 
not  done  the  task  or  if  you  have  not  seen  him  do  it. 
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SEC'xivJN  2 


Is  he  capable  of  performing  proficiently  the 
equipment  turn-on,  warm-up,  run,  and  shut¬ 
down  procedures  involved  in 


Yes  No 


1.  Operating  avionic  equipments 

2.  Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 

3.  Making  out  reports  (failure,  etc. ) 

4.  Performing  postflight  inspections  of  avionic 
equipments 

5.  Standing  watch 

6,  Performing  preflight  inspections  of  avionic 
equipments 

7.  Replacing  repaired  avionic  parts  /equipment 
in  planes 

8.  Performing  routine  line  operations 

9.  Performing  preventive  maintenance  on 
avionic  equipments 

10.  Removing  malfunctions  avionic  parts/equip¬ 
ment  from  planes 

1 1 .  Performing  intermediate  inspections  of 
avionic  equipments 

12.  Performing  minor  inspections  of  avionic 
equipments 
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Directions  for  Section  3 


Section  3  contains  9  ta3ks  which  vary  in  terms  of  the  extent  to  which 
they  involve  removal  of,  making  required  corrections  in,  and  replacement  of 
avionic  equipment. 

Think  of  the  man  you  are  evaluating.  Is  he  capable  of  performing  pro¬ 
ficiently  the  removal  of,  making  required  corrections  in,  and  the  replacement 
of  avionic  equipment  involved  in  the  first  task  Mon  his  own"  without  direct  super¬ 
vision  ?  Make  a  check  mark  in  either  the  '^Yes*1  or  ''No'*  box  next  to  the  task, 
according  to  your  best  judgment.  Continue  on  down  the  page.  Remember  to 
evaluate  the  man  ONLY  with  respect  to  whether  or  not  he  is  capable  of  perform- 
ing  the  removal  of,  making  required  corrections  in,  and  the  replacement  of 
avionic  equipment  involved  in  that  task  "on  his  own"  without  direct  supervision. 
Be  sure  to  make  a  check  mark  next  to  each  of  the  9  tasks. 
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SECTION  3 


Is  he  capable  of  performing  proficiently  the 
removal  of,  making  required  corrections  int 
and  the  replacement  of  avionic  equipment 
involved  in 


i 


Yes  No 


1.  Performing  intermediate  inspections  of 
avionic  equipments 

2.  Performing  routine  line  operations 

3.  Performing  preflight  inspections  of  avionic 
equipments 

4.  Operating  avionic  equipments 

5.  Performing  preventive  maintenance  on 
avionic  equipments 

6.  Performing  minor  inspections  of  avionic 
equipments 

7.  Performing  postflight  inspections  of  avionic 
equipments 

8.  Standing  watch 

9.  Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 
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Directions  for  Section  4 


This  time  the  13  tasks  listed  in  Section  4  are  to  be  viewed  from  the 
standpoint  of  the  safety  precautions  that  they  involve. 

Follow  the  same  procedures  you  did  in  the  previous  sections.  Ask 
yourself  this  question  about  the  man  you  are  evaluating:  Is  he  capable  of  per¬ 
forming  the  safety  precautions  involved  in  the  first  task  ^on  his  own11  without 
direct  supervision?  Place  a  check  mark  in  the  appropriate  box.  Ask  the  same 
question  with  regard  to  the  other  tasks  in  the  list  and  make  an  appropriate  check 
mark  next  to  each  task  according  to  your  answer  to  the  question.  Remember, 
you  should  make  your  evaluation  ONLY  with  respect  to  whether  or  not  he  is  capa¬ 
ble  of  performing  the  safety  precautions  involved  in  each  task. 
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SECTION  4 


Is  he  capable  of  performing  the  safety 
precautions  involved  in 


Yes  No 


1 .  Following  block  diagrams  for  avionic 
equipments 

2.  Operating  avionic  equipments 

3.  Performing  minor  inspections  of  avionic 
equipments 

4.  Performing  preventive  maintenance  on 
avionic  equipments 

5.  Replacing  repaired  avionic  parts/ equipment 
in  planes 

6.  Performing  routine  line  operations 

7.  Operating  standard  test  equipment  for  deter¬ 
mining  malfunctions  in  avionic  equipments 

8.  Performing  posKlight  inspections  of  avionic 
equipments 

9.  Performing  preflight  inspections  of  avionic 
equipments 

10.  Performing  intermediate  inspections  of 
avionic  equipments 

11.  Removing  malfunctioning  avionic  parts  /equip¬ 
ment  from  planes  ^ 

12.  Making  out  reports  (failure,  etc.) 

13.  Standing  watch 
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